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Building Mathematical Identity After School: Year 1 of a Cluster-Randomized Trial 


Background 

We are conducting a cluster-randomized trial of an after-school program intended to build 
mathematical identity among students in groups under-represented in STEM. Populations 
under-represented in STEM include women, persons with disabilities, African Americans, and 
Latino/as (NSF, 2011). U.S. jobs are growing most rapidly in areas that require STEM knowledge 
(National Academy of Sciences, 2010). However, over half of secondary students do not feel 
they need math outside of school, and they think that liking math is unpopular (Markow & 
Moore, 2001). If these students continue along this path, they will not be eligible for at least 
75% of the jobs of the future (Fleming, 2012). Elementary-level experiences are thus important 
in attracting students to STEM. 


Mathematical identity involves the ways that students think about themselves in relation to 
mathematics and the extent to which they have developed a commitment to, and have come to 
see value in, mathematics (Cobb, Gresalfi, & Hodge, 2009). Therefore, it encompasses 
persistence and interest in mathematics and motivation to learn mathematics. A strong 
mathematical identity is thought to be key to doing well and persisting in math. Out-of-school- 
time experiences may build positive math identity in struggling students because pressure to 
perform for grades and scores is absent. This could translate into higher achievement in school 
as well. After-school programs may increase students’ access to mathematical content and 
discourse and build identity as knowers/doers of mathematics (Cobb & Gresalfi, 2006). With 
some exceptions (Berry, 2008; Nasir, 2002), few previous studies examine identity motivation, 
and fewer yet study out-of-school experiences and identity motivation. 


Research Questions 

1. What is the effect of the after-school program on students’ math identity, engagement, 
interest, and math achievement? 

2. What is the relationship between math achievement and math engagement, interest, and 
identity in the after-school program? 

3. What elements of the program foster students’ math identity, and through what 
mechanisms is this effect achieved? What are the best practices and lessons learned? 


Setting 
We are implementing the study in 45 after-school programs that serve students in grades 4-5. 


Intervention 

After-School Math (ASM) is designed to provide engaging, inquiry-based math activities that 
enhance the key influencers of math identity (see theory of action in Exhibit 1). It has been 
implemented in diverse sites across the country. Students are exposed to 4 themes: ArtMath, 
Jump Rope Math, Built Environment, and MusicMath, which take around 12 weeks each to 
implement. There is one day of educator training for each theme and follow-up support. 


Research Design 
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Programs (minimum 20 students and 2 educators each) are randomly assigned to implement 
the full ASM curriculum or a control version without activities hypothesized to foster math 
identity. Implementation of the first two themes occurred in 2016-17, with the last two planned 
for 2017-18. 


Data Collection 
Data sources include: 

Student survey of math identity. This pre/post survey measures students’ math identity 
(see Exhibit 2) and gathers data on students’ reactions to the ASM curriculum activities. It was 
administered at baseline and after each theme. 

Training feedback form. This form collects after-school educators’ immediate feedback 
on their experiences with the professional development intended to prepare them to lead each 
theme of ASM. It was administered after training for each theme. 

Educator survey. This survey captures which activities they completed, challenges, 
supports for the implementation, and the perceived effect of ASM on students, as well asa 
version of the survey of math identity that was adjusted to be appropriate for adults who were 
no longer in school. It was administered after each theme was completed. 

Observation protocol. The study team adapted the STEM version of the Youth Program 
Quality Assessment (STEM PQA). It assesses the extent to which activities engage students, 
build skills, and provide meaningful opportunities to foster math identity. We will conduct 
observations in 7 treatment and 5 control sites in 2017-18. 

Educator interview protocol. The interview questions are designed to surface the 
educator's implementation of the program, as well as perspective on its influence on student 
math identity. It also include items about educators’ own perceptions of math identity. 

Achievement. We will obtain student pre- and post-program math achievement test 
scores. 


Analysis 

We have analyzed all Year 1 data descriptively and will conduct impact analyses in fall 2017. The 
impact analyses will use a 2-level model. At level 1, we will control for student-level 
characteristics (baseline math identity, prior achievement, demographics). At level 2, we will 
control for program covariates. We will also conduct subgroup analyses to look at impacts for 
girls and minority students. 


Findings 

Findings from the first year of implementation indicate several challenges. Some sites did not 
send the required number of staff or did not attend the training. For example, of the 24 
programs enrolled at the time of the first training, 2 dropped out of the study before training, 
and 9 were trained but never implemented, largely due to losing overall program funding and 
not being able to operate. Several educators reported that they did not implement every 
activity. Sites that implemented the first theme completely were more likely to complete 
second theme. Treatment and control sites had similar levels of completion for each theme 
(range = 71% - 85% of activities), but some treatment sites skipped the treatment-only 
activities. As is common in after-school environments, students frequently left, joined, or 
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rejoined the afterschool program. Initial (unadjusted) analyses from the ArtMath pilot theme 
showed no treatment-control difference in their math identity, although treatment students 
were more likely say that they had fun and would participate in more activities. Treatment 
educators reported higher self-perception of math ability after the Built Environment theme. 
The full paper in spring 2018 will present full, adjust results of impact analyses. 


Conclusions 

This paper will present first-year findings from an RCT of an afterschool curriculum designed to 
improve students’ math identity. Preliminary analyses indicate no impact on student outcomes 
and some impact on educator outcomes. There were substantial challenges to implementation. 
Fall 2017 will include adjusted impact estimates and psychometric analyses of the math identity 
instruments. 


SREE Spring 2018 Conference Abstract 4 


References 


Berry, R. Q. (2008). Access to upper-level mathematics: The stories of African American middle 
school boys who are successful with school mathematics. Journal for Research in 
Mathematics Education, 39(5), 464-488. 


Cobb, P., & Gresalfi, M. (2006). Cultivating students’ discipline-specific dispositions as a critical 
goal for pedagogy and equity. Pedagogies: An International Journal, 1(1), 49-57. 


Cobb, P., Gresalfi, M., & Hodge, L. (2009). "An interpretive scheme for analyzing the identities 
that students develop in mathematics classrooms." Journal for Research in Mathematics 
Education: 40-68. 


Fleming, J. (2012). Enhancing minority student retention and academic performance: What we 
can learn from program evaluations. Hoboken, NJ: Jossey-Bass/Wiley. 


Markow, D. & Moore, K. “Progress Toward Power: A follow-up survey of children and parents 
attitudes about math and science.” NACME Research Letter 9(1). 


Nasir, N. S. (2002). Identity, goals, and learning: Mathematics in cultural practice. Mathematical 
Thinking & Learning, 4(2/3), 213-247. 


National Academy of Sciences (2010). Rising above the gathering storm, revisited: Rapidly 
approaching category 5. Washington, DC: The National Academies Press. 


National Science Foundation. (2011). Women, minorities, and persons with disabilities in science 
and engineering. Washington, DC: Author. 


SREE Spring 2018 Conference Abstract A-1 


Exhibit 1: Revised Theory of Action 


ACTIVITIES SHORT-TERM INTERMEDIATE LONG-TERM 
OUTCOMES OUTCOMES OUTCOME 


Professional After-school educator Educators’ improved: 
development and knowledge and self- ¢ Math identity 
technical assistance for y efficacy for building ¢ Intention to involve 
after-school educators math identity in youth math in after-school 


Program context factors: funding stability, infrastructure, administrative support for implementation 
Individual context factors: attitudes, dispositions, and previous experiences in math 


Exhibit 2. Student Math Identity Survey — Subscales, Sources, and Samples 
Source Sample Item 


Modified Fennema-Sherman Math Math is a worthwhile, necessary 
Attitude Scale - Usefulness (Mulhern & — |subject. 


i | am really good at math. 
math ability 


Math enjoyment Math is fun. 


Member of math _|Researcher-developed People like me do math. 
Future math plans |Researcher-developed | plan to use math in my future 
career. 


Growth mindset [Becoming Effective Learners Survey My intelligence is something | 
(Farrington, Levenstein, & Nagaoka, can’t change very much. 
2013) (reversed) 
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